Skip to content

Cleaning up potential resource handling errors.#1258

Merged
rma-rripken merged 8 commits into
developfrom
bugfix/closing_streams
Sep 5, 2025
Merged

Cleaning up potential resource handling errors.#1258
rma-rripken merged 8 commits into
developfrom
bugfix/closing_streams

Conversation

@rma-rripken
Copy link
Copy Markdown
Collaborator

Explicitly calling blob and clob free. Putting streams from blob and clob in try-with-resources blocks. Calling jOOQ fetchStream in try-with-resources.

I don't think any of these is actually causing connection leaks but they seem like easy changes for better safety

@rma-rripken rma-rripken marked this pull request as ready for review August 27, 2025 19:30
@rma-rripken
Copy link
Copy Markdown
Collaborator Author

@MikeNeilson Not immediately obvious to me why this build is failing

@MikeNeilson
Copy link
Copy Markdown
Contributor

@MikeNeilson Not immediately obvious to me why this build is failing

https://github.com/USACE/cwms-data-api/pull/1258/files#annotation_38309067931

Granted that still doesn't make it obvious. All of those errors should be in a log in the build directory, we should probably upload it as an action artifact that so we can review it.

@rma-rripken
Copy link
Copy Markdown
Collaborator Author

Its hit-or-miss for me whether that annotations link shows anything of not. The github ui seems to be inconsistent.

@rma-rripken
Copy link
Copy Markdown
Collaborator Author

I've added to the pr and I believe it will fix #1252

@MikeNeilson
Copy link
Copy Markdown
Contributor

Its hit-or-miss for me whether that annotations link shows anything of not. The github ui seems to be inconsistent.

Yeah, it tends not to show on the file unless it's a file that was edited and shows in the PR.

@MikeNeilson
Copy link
Copy Markdown
Contributor

It's failing on the Time Series integration test that inserts a massive amount of data.

@rma-rripken
Copy link
Copy Markdown
Collaborator Author

Something has changed in the backend. I ran the timeseries test_create_big that inserts 200,000 points and the connection gets forceably closed in 45 seconds and the insert still hasn't completed. I'm glad the test is failing if its taking that long but the test used to pass. Is it the change that we are now actually closing the connections? Dialing down the number of points to 10,000 and the zstore_ts method is still taking over 30s to complete. And that is after I moved the json serialization (400ms) and ZTSV array creation (30ms) so it happens before we get a connection to do the zstore_ts call. Maybe I create a new issue to figure out why the timeseries store is so slow?

@MikeNeilson
Copy link
Copy Markdown
Contributor

Something has changed in the backend. I ran the timeseries test_create_big that inserts 200,000 points and the connection gets forceably closed in 45 seconds and the insert still hasn't completed. I'm glad the test is failing if its taking that long but the test used to pass. Is it the change that we are now actually closing the connections? Dialing down the number of points to 10,000 and the zstore_ts method is still taking over 30s to complete. And that is after I moved the json serialization (400ms) and ZTSV array creation (30ms) so it happens before we get a connection to do the zstore_ts call. Maybe I create a new issue to figure out why the timeseries store is so slow?

Yeah, the only recent change I can think of was for retrieve data entry, at least within CDA, but it's definitely possible a database change screwed something up.

@rma-rripken rma-rripken merged commit a660ea5 into develop Sep 5, 2025
6 of 9 checks passed
@rma-rripken rma-rripken deleted the bugfix/closing_streams branch September 5, 2025 21:36
MikeNeilson pushed a commit that referenced this pull request Oct 23, 2025
Explicitly calling blob and clob free. Putting streams from blob and
clob in try-with-resources blocks. Calling jOOQ fetchStream in
try-with-resources.

I don't think any of these is actually causing connection leaks but they
seem like easy changes for better safety

(cherry picked from commit a660ea5)
MikeNeilson pushed a commit that referenced this pull request Oct 24, 2025
Explicitly calling blob and clob free. Putting streams from blob and
clob in try-with-resources blocks. Calling jOOQ fetchStream in
try-with-resources.

I don't think any of these is actually causing connection leaks but they
seem like easy changes for better safety

(cherry picked from commit a660ea5)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants